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ABSTRACT 


Neural  networks  have  demonstrated  the  potential  to  deal 
with  problems  related  to  human  cognition  that  to  date  have 
stymied  researchers:  problems  that  traditional  rule  and 
logic  based  artificial  intelligence  techniques  have  been 
unable  to  solve. 

The  range  of  possible  neural  network  applications  is 
not  yet  fully  understood.  To  date,  research  and  development 
has  tended  to  concentrate  on  a  most  impressive  array  of 
real-time  embedded  systems  such  as  speech  processors,  target 
recognition  and  robotics  control  systems.  The  Department  of 
Defense  is  a  strong  proponent  of  this  technology. 

There  is  another  side  to  the  development  and  practical 
use  of  neural  network  applications.  Inexpensive  software  is 
presently  available  that  allows  virtually  any  personal 
computer  to  function  as  a  simulated  neurocomputer. 

This  thesis  -  presented  in  a  personal  computer  based 
hypertext  environment  (a  diskette  is  available)  -  reports 
the  results  of  an  exhaustive  literature  search  and  extensive 


hands-on  experience  with  these  tools  and  techniques. 
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THESIS  DISCLAIMER 


The  reader  is  cautioned  that  the  computer  programs  and 
procedures  described  in  this  research  have  not  been 
exercised  for  all  cases  of  interest.  While  every  effort  has 
been  made,  within  the  time  available,  to  ensure  that  they 
are  free  of  errors  in  concept,  design  and  implementation, 
they  cannot  be  considered  validated.  Any  application  of 
these  methods  is  at  the  risk  of  the  user. 

The  diskette  containing  the  hypertext  thesis  may  be 
copied  for  distribution  provided  that  credit  is  given  to  the 
author.  Due  to  copyright  restrictions,  the  BrainMaker™ 
software  -  a  backpropagation  based  neural  network  -  is  not 
included  on  the  diskette.  It  is  required  only  for  the 
demonstration  section  of  the  thesis. 
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I .  INTRODUCTION 


A.  BACKGROUND 

This  thesis  is  the  result  of  an  exhaustive  literature 
search  coupled  with  extensive  hands-on  experience  with  many 
of  the  methods  described.  Despite  the  fact  that  the  seminal 
theories  surrounding  neural  networks  were  published  over 
thirty  years  ago,  "connectionism”  is  an  immature,  but  very 
vibrant,  inter-disciplinary  field.  It  encompasses  cognitive 
theory,  decision  theory,  logic,  computer  science, 
mathematics,  statistics,  operational  analysis  and  more. 
Neural  networks  have  the  potential  to  deal  with  problems 
related  to  human  cognition  that  to  date  have  stymied 
artificial  intelligence  researchers.  The  intent  of  this 
thesis  is  not  however,  to  wantonly  proselytize  the  virtues 
of  neural  network  techniques,  but  rather  to  inform  a  diverse 
readership  of  their  characteristics  and  to  encourage  further 
experimentation . 

B.  SCOPE  AND  LIMITATIONS 

Due  to  heightened  interest,  the  corpus  of  theoretical 
information  pertaining  to  neural  networks  is  exploding .  New 
methods  are  published  monthly  and  older  ones  are  continually 
being  updated,  refined  and  augmented.  It  is  impossible  to 
accurately  synopsize  such  a  volatile  target.  The  idea  of 
mimicking  the  function  of  the  brain  is  by  itself 
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fascinating,  but  even  in  its  embryonic  phase,  this 

technology  is  capable  of  addressing  some  vexing  real  world 

problems.  These  problems  range  from  the  control  of 

supersonic  aircraft  to  gleaning  information  from  diverse  and 

complex  data.  It  is  in  this  latter  domain  that  this  paper 

will  focus.  A  variety  of  diverse  methodologies  will  be 

discussed,  but  only  one  problem  solving  approach  will  be  i 

examined  in  detail.  Though  personal  computer  based  neural 

« 

network  simulations  are  not  as  fast  as  networks  implemented 
directly  in  silicon,  they  are  capable  of  supporting  the 
efficient  solution  of  problems  that  do  not  yield  to 
statistical  regression  or  optimization.  In  the  course  of 
this  thesis  a  representative  sample  problem  will  be 
undertaken  and  the  reader  will  be  afforded  the  opportunity 
to  interactively  participate  in  its  solution.  This  thesis 
is  intended  to  stimulate  further  research  and  exploitation 
of  these  techniques,  but  it  cannot  hope  to  provide  an 
authoritative  procedural  approach  to  their  use. 

« 
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I I .  ORGANI Z ATION 


A.  WHY  HYPERTEXT? 

This  typewritten  paper  document  serves  only  to 
introduce  and  explain  the  motivations  and  methods  behind  the 
actual  thesis,  which  has  been  written  in  a  PC-based 
hypertext  medium  (the  diskette  is  available).  Hypertext  was 
deemed  appropriate  for  the  following  reasons: 

-it  enhances  the  comprehensibility  of  the  material; 

-it  appeals  to  a  broad  range  of  readers; 

-it  efficiently  integrates  text  and  graphics; 

-it  provides  for  an  actual  working  demonstration  of  the 
neural  network  methods  described; 

-it  simplifies  duplication,  distribution  and  storage 
of  the  document. 

In  order  to  understand  the  uniqueness  and  versatility 
of  hypertext  methods  it  will  prove  useful  to  contrast  them 
with  more  familiar  forms  of  printed  text.  This  section 
briefly  highlights  the  benefits  of  using  hypertext.  The 
reader  interested  in  hypertext  should  examine  Reference  1. 

Traditionally  text  is,  by  its  nature,  sequential.  The 
author  assumes  the  responsibility  for  defining  the  order  in 
which  his  work  will  be  read.  Tht  reader  proceeds 
systematically  from  word  to  word,  sentence  to  sentence,  and 
page  to  page  until  finished.  Such  a  document  is  both 
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logically  and  physically  linear.  In  contrast,  hypertext  is 
non-sequential .  It  not  only  permits,  but  also  encourages 
the  reader  to  decide  the  sequence  that  suits  him.  This  is 
not  altogether  a  new  idea,  as  manual  forms  of  hypertext  have 
existed  for  centuries.  An  encyclopedia  is  a  good  example. 

I 

Each  article  contains  references  to  other  related  articles 
offering  the  reader  the  option  to  follow  a  "link”  to  i 

peripheral  information.  In  this  case,  navigation  is  part  of 
the  reading  process. 

A  computer  running  appropriate  software  can  readily 
implement  these  hypertext  techniques.  Using  just  the 
keyboard  (or  other  input  devices),  the  reader  may  meander 
through  a  document  at  his  discretion,  assimilating  the 
material  in  an  order  and  context  that  suit  him.  To  prevent 
the  reader  from  getting  disoriented,  navigational  tools  are 
in  integral  part  of  the  document.  He  can  follow  a 
prearranged  tour,  locate  his  present  position  in  the 
document  on  a  graphic  map,  and  recall  a**  1  of  the  steps  that 
he  has  taken,  so  that  he  can  ascertain  where  he  has  been  and 
how  much  of  the  document  he  has  read.  * 

It  is  not  difficult  to  see  the  appeal  of  such  a  system: 
both  the  author (s)  and  the  readers  are  afforded  considerable 
flexibility.  A  properly  designed  hypertext  document  is 
capable  of  imparting  at  least  as  much  information  as 
conventional  text  and  probably  a  lot  more. 
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Granted,  this  technique  has  technological,  practical 
and  aesthetic  advantages,  but  in  many  situations  it  is 
simply  the  most  appropriate  way  to  present  the  material. 
Perhaps  more  importantly,  due  to  its  compel lingly 
interactive  nature,  readers  tend  to  enjoy  it.  [Ref.  1] 
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B.  HYPERWRITER!™  SPECIFICS 

The  main  portion  of  this  thesis,  contained  on  the 
available  computer  diskette,  was  written  using  a 
commercially  available  hypermedia  authoring  software 
entitled  HyperWriter !™.  This  authoring  system  offers  the 
developer  the  ability  to  integrate  text,  graphics,  video  and 
audio  as  well  as  hypertext  "links"  into  a  single  document.  I 

The  result  of  using  HyperWriter!™  on  a  body  of  information 

% 

is  an  interactive  document  that  a  user  can  explore  in  a  non¬ 
linear  fashion. 

For  the  author,  this  represents  a  departure  from 
traditional  techniques  of  composition.  Disparate  elements 
can  be  located  where  appropriate,  and  the  reader  can  be 
provided  access  to  this  information  along  several  very 
different  paths.  Composing  a  document  using  these  methods 
has  a  decidedly  different  feel  to  it,  one  acknowledges  the 
responsibility  to  not  only  inform  the  reader,  but  to  guide 
him. 

The  HyperWriter!™  software  contains  features  necessary 
for  the  creation  of  attractive  and  effective  hypertext 
documents.  It  includes  a  text  editor  with  a  built  in 

f 

spelling  checker,  and  the  ability  to  import  pre-existing 
text  and  graphics  files.  Most  importantly,  its  companion 
"reader"  software,  HyperReader™ ,  is  licensed  for  unlimited 
distribution  at  no  cost  to  the  reader,  much  simplifying  the 
process  of  distributing  the  document.  [Ref. 2] 
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C.  DESIGN  CONSIDERATIONS 

Despite  the  freedom  of  choice  that  hypertext  affords 
the  reader,  to  some  extent  the  author  maintains  the  ability 
to  influence  the  manner  in  which  the  document  is  read.  If 
the  author  deems  it  important  for  the  reader  to  view  a 
particular  portion  of  the  text  in  its  entirety,  then  it 
should  be  presented  in  that  way.  On  the  other  hand,  the 
reader  can  and  should  be  encouraged  to  explore  when  sequence 
or  cohesiveness  is  less  vital.  The  reader's  background 
knowledge  of  the  subject  area  must  be  taken  into  account  so 
that  terminology  can  be  used  without  the  need  to  embed 
detailed  explanations.  Any  confusing  or  ambiguous  terms  can 
be  defined  through  a  link,  thus  allowing  the  knowledgeable 
reader  to  avoid  the  tedium  of  reading  descriptions  of 
familiar  terms,  while  the  uninformed  reader  has  free  and 
ready  access  to  those  descriptions.  This  in  turn  effects 
the  length  of  any  given  portion  of  the  document.  The 
individual  portions  can  be  made  much  more  terse  while  still 
conveying  all  necessary  information.  Properly  executed,  the 
same  information  can  appeal  to  a  far  wider  readership. 

Every  effort  must  be  made  to  establish  meaningful 
conventions  and  to  adhere  to  them  consistently.  In  this 
thesis  all  links  to  graphic  images  are  colored  green,  all 
references  are  red  and  any  item  with  a  rectangular  box 
surrounding  it  leads  to  a  link.  In  a  document  of  this  kind 
the  reader  will  generally  be  presented  with  an  overwhelming 
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amount  of  new  information,  consequently  his  interface  to  the 
material  should  become  transparent  to  him  and  should  not  be 
allowed  to  intrude  or  to  disconcert  him.  Figure  1 
graphically  illustrates  the  simplicity  of  the  hypertext 
interface . 


TYPICAL  HYPERTEXT  SCREEN 

i.e 

History  of  Noursl  HetMorks- 

2.6 

Neurol  Network  Nethodologles 

3.6 

Neural  Networks  and  the  DOO 

4.6 

6  Neural  Decision  Support  Systee 

5.6 

Conclusions: 

6.6 

Bibliography 

1  I 

j  j 

Pillil' 

Figure  1  A  typical  hypertext  screen 


In  the  interest  of  conventionality,  this  hypertext 
document  has  been  constructed  to  appear,  at  least  in  part, 
as  a  traditional  typewritten  research  paper.  Essentially, 
this  means  that  the  text  is  presented  in  a  fairly  linear 
fashion  and  all  of  the  linkages  extend  from  the  text. 
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A  more  powerful  alternative  may  be  to  organize  the 
material  visually,  through  the  use  of  on-screen  graphical 
images  or  maps.  The  links  extending  from  these  graphics 
would  lead  the  reader  to  accompanying  explanatory  text. 
This  method  proved  effective  in  some  areas  of  this  thesis, 
but  the  underlying  design  philosophy  was  generally  to 
emphasize  links  from  text  to  graphics. 
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D.  SYSTEM  REQUIREMENTS 

This  hyper-document  is  designed  to  be  read  on  a 
typically  configured  personal  computer. 

Specifically  the  requirements  include; 

-IBM  Disk  Operating  System  (DOS)  version  3.0  or  later 
-a  color  monitor  supporting  Expanded  Graphics  Array 
(EGA)  or  Video  Graphics  Array  (VGA) 

-a  hard  disk  drive  with  approximately  2.5  megabytes  of 
available  storage  space 
-a  Microsoft™  compatible  mouse 

-a  minimum  of  520  kilobytes  of  available  system  memory 


E.  INSTALLATION 

The  distribution  diskette  is  available  free  of  charge. 

If  you  desire  a  copy  please  mail  a  formatted  high  density 

diskette  along  with  sufficient  return  postage  to: 

Prof.  Tung  Bui,  Code  AS/BD 
Naval  Postgraduate  School 
Monterey,  CA  93943-5000 

An  automated  utility  has  been  created  to  make  the 
installation  of  this  document  as  trouble  free  as  possible. 

It  is  designed  to  copy  the  files  from  a  high  density  (either 
5  1/4  inch  1.2  megabyte  or  3  1/2  inch  1.44  megabyte)  floppy 
disk  drive  labeled  A:  to  the  hard  disk  drive  labeled  C: . 

Tq  ins.ta.II: 

-insert  the  floppy  disk  in  the  A:  drive 
-type  A:  and  press  Enter 
-type  INSTALL  and  press  Enter 

-the  program  will  instruct  you  to  type  START  and  press 
Enter  to  begin  reading 

Note:  If  your  system's  disk  drives  are  not  labeled  in  this 
way  you  must  manually  install  the  software.  This  will 
require  you  to  create  a  subdirectory  on  the  chosen  hard 
disk,  copy  all  of  the  files  from  the  floppy  diskette  to  that 
subdirectory,  decompress  the  files  and  finally  type  START. 
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III.  NEURAL  NETWORK  DECISION  SUPPORT  FOR  THE  DOD 

A.  SUMMARY  (HYPERTEXT  VERSION) 

Neural  network  methods  are  simply  too  enticing  to 
ignore.  Despite  their  current  popularity  among  some 
academics  and  much  of  industry,  their  widespread  use  is  at 
best  still  a  few  years  away.  The  groundwork  has  been  laid, 
but  further  research  must  contend  with  a  significant  number 
of  theoretical  and  practical  issues.  Nascent  neural  systems 
have  been  denounced  as  being  "toy”  solutions  to  "toy" 
problems.  In  part  this  is  a  valid  criticism.  At  this 
evolutionary  stage,  most  real-world  problems  remain  slightly 
out  of  reach.  This  is  at  least  partially  due  to  the  fact 
that  real-world  data  is  rarely  maintained  in  a  form  readily 
suitable  for  neural  network  use.  At  issue  too,  are  the 
networks'  ability  to  grow  in  size  and  complexity,  because 
most  of  the  current  architectures  slow  down  unacceptably 
when  presented  with  complex  problems.  Research  in  the  area 
of  hybrid  networks  is  in  its  infancy,  but  it  certainly 
offers  tremendous  promise  for  dealing  with  increasingly 
complex  cognitive  problems. 

Another  area  of  concern  must  be  the  hyperbole  that 
surrounds  neural  networks.  Virtually  all  of  the  commercial 
software  simulation  products  are  guilty  of  making  some 
pretty  outlandish  claims.  There  is  a  very  real  danger  that 
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the  technology  will  be  oversold  and  will  subsequently 
disappoint  and  frustrate  its  user  base.  Rule  based  expert 
systems  have  set  the  historical  precedent.  The  general 
population  has  been  promised  "thinking"  computers  for  more 
than  thirty  years;  the  mounting  disappointment  has  created  a 
generation  of  skeptics.  Perhaps  this  is  a  blessing  in 
disguise.  If  neural  networks  survive  this  skepticism  and 
scrutiny,  they  will  be  more  likely  to  emerge  as  mature  and 
useful  systems. 

There  are  immediate  dangers  if  poorly  designed  and 
inadequately  tested  neural  network  applications  are  rushed 
into  service.  Such  systems  may  glibly  output  dangerously 
incorrect  answers.  The  question  of  legal  liability  has 
already  been  raised  with  regard  to  expert  systems.  Ethical 
questions  will  also  arise  regarding  the  invasion  of  personal 
privacy.  Thoughtful  policy  guidelines  and  even  legislation 
will  eventually  be  required.  In  any  event,  a  thoroughly 
conservative  approach  should  be  applied.  Such  an  approach 
will  demand  both  rigor  and  patience,  but  will  provide  the 
most  lasting  benefits. 

In  order  for  this  technology  to  flourish,  the  future 
user  population  must  be  given  a  rudimentary  education  in  its 
inherent  capabilities.  This  will  serve  two  purposes.  First, 
it  will  sensitize  potential  users  to  the  type  and  format  of 
data  that  is  most  usable  by  neural  network  applications.  No 
problem  can  be  solved  by  a  neural  network  if  the  relevant 
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data  is  insufficient.  With  a  little  bit  of  foresight,  all 
future  databases  will  be  ripe  for  neural  analysis.  Secondly, 
if  the  data  exists  in  the  proper  form,  the  users  will  clamor 
for  new  analytical  tools. 

In  this  era  of  federal  fiscal  austerity  it  may  be 
necessary  to  divert  research  funding  from  other  programs, 
perhaps  even  from  some  of  the  expensive  real-time  neural 
systems.  Due  to  the  almost  unlimited  spectra  of 
applications,  it  may  be  possible  to  "piggyback"  neural 
efforts  onto  existing  projects.  From  a  contractual  point  of 
view  this  would  be  desirable,  because  existing  government 
software  contracting  methods  don't  readily  apply  to  this 
technology.  [Ref.  3] 

The  Department  of  Defense  can  and  should  continue  to 
aggressively  pursue  neural  network  research  with  an  emphasis 
on  deployable  systems.  The  conceivable  benefits  of  this 
leadership  are  enormous. 

The  POD  should; 

-Adopt  (or  create)  a  standard  neural  network 
development  package  suitable  for  the  average  personal 
computer  user 

-Formalize  the  application  development  cycle 

-Create  and  man  roving  neural  "tiger  teams"  with  a 
charter  to  seek  out  potential  application  areas 

-Establish  an  on-line  neural  "clearinghouse"  using  the 
Defense  Data  Network  (DDN) 

-Freely  grant  access  to  data  for  research  purposes  and 
make  it  available  for  transfer  over  the  DDN 
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-Maintain  existing  expert  systems  for  possible 
incorporation  into  hybrid  systems 

-Completely  rewrite  DOD-STD  2167A,  to  make  contracting 
for  the  development  and  purchase  of  these  applications 
feasible 

-Examine  this  technology  as  a  remedy  for  the  ongoing 
software  "crisis"  because  it  involves  no  conventional 
(and  expensive)  programming 

-Most  importantly,  begin  educating  portent iaL  users 
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IV.  CONCLUSIONS 


The  promise  of  neural  network  approaches  to  complex 
real  world  problems  has  not  gone  unnoticed.  Both  the 
government  and  the  civilian  sector  are  vigorously  pursuing 
its  application.  When  compared  to  other  areas  of  research, 
the  level  of  funding  reguired  is  minuscule.  There  are  no 
guarantees  of  success,  but  the  most  relevant  risk  is  not 
related  to  failure,  it  is  failing  to  participate  in  future 
successes. 

As  a  by-product  of  this  research,  it  is  clear  that  the 
hypertext  authoring  environment  has  proven  itself  ideally 
suited  for  publications  of  this  kind.  The  commercial 
software  tools  currently  available,  in  conjunction  with  the 
general  proliferation  of  personal  computers,  make  this 
approach  eminently  practical. 
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